Using tactile and visual sensing with a robotic hand

Most robotic hands are either sensorless or lack the ability to accurately and robustly report position and force information relating to contact. This paper describes a robotic hand system that uses a limited set of native joint position and force sensing along with custom designed tactile sensors and real-time vision modules to accurately compute nger contacts and applied forces for grasping tasks. Three experiments are described: integration of real-time visual trackers in conjunction with internal strain gauge sensing to correctly localize and compute nger forces, determination of contact points on the inner and outer links of a nger through tactile sensing and visual sensing, and determination of vertical displacement by tactile sensing for a grasping task.